第一題 [分類器、特徵建構、與Stacking]¶
(50%) 建構分類器時常會透過特徵選擇與Ensemble增強預測能力。本題的目的是讓大家練習這些技巧。本題使用一個中文姓名分類的資料集。這個資料集收集了10730個中文名(不含姓)，以及這些名子對應到的性別。本資料集的資料檔是namesex_data_v2.csv，共有三個欄位，gname, sex, fold:
- gname: 不含姓的中文名。
- sex: 1為男性，0為女性。
- fold: 為0 - 9的整數，做為後續切割資料集使用。
我們先將這個資料集讀入Numpy Array，並列印前十筆資料。
%matplotlib inline
import matplotlib.pyplot as plt
import csv
import numpy as np
import warnings
warnings.filterwarnings("ignore")
f = open('ds/namesex_data_v2.csv', 'r', encoding='utf8')
mydata = csv.DictReader(f)
sexlist = []
namelist = []
foldlist = []
for i, arow in enumerate(mydata):
if i < 10:
print(arow)
sexlist.append(int(arow['sex'].strip()))
gname = arow['gname'].strip()
namelist.append(gname)
foldlist.append(int(arow['fold'].strip()))
sexlist = np.asarray(sexlist)
namelist = np.asarray(namelist)
foldlist = np.asarray(foldlist)
f.close()
接下來看一些統計數字:
nobs = namelist.shape[0]
print("資料筆數 = ", nobs)
avg_sex = np.mean(sexlist)
print("男性比率", avg_sex)
這個資料集的男女比率還算是平均，男性佔了47.94%。
我們用了比較迂迴的方式讀檔案，Pandas其實可以直接讀csv檔。接下來做一些基本的資料分析。我們關心幾件事:
- 最常見的名子
- 共有多少個不重複的名子
- 有多少難判斷男女的名子
要回答這些問題，首先對名子加總，計算數量與男性比率，排序之後列出前20個名子:
import pandas as pd
name_df = pd.DataFrame({'sex': sexlist, 'name': namelist, 'fold': foldlist})
sex_tend = name_df[['name', 'sex']].groupby(["name"]).agg(['mean', 'count']).reset_index()
sex_tend.columns = ['-'.join(col).strip() for col in sex_tend.columns.values]
sex_tend = sex_tend.sort_values(['sex-count'], ascending=False)
print("最常見的20個名子:")
sex_tend.head(20)
由上面的列表看來，最常見的男性名子是承恩與冠廷，最常見的女性名子是宜蓁與佳穎。而這些常見的名子男性比率都非常接近0或1，表示這些名子沒有性別混淆的問題。
另外一個問題是有多少名子只出現一次:
single_name = np.sum(sex_tend['sex-count'] <= 1)
print("出現一次名子個數: ", single_name, "比率:", single_name / nobs)
由結果看來，名子在資料集中只出現一次的比率很高，有65%。也就是說，如果單純的使用訓練資料中出現的名子做為特徵，模型應該會有很差的預測能力。
為了處理這個問題，我們在接下來的任務中，會使用"Unigram + Full Name"的特徵表示，比如說，"芳瑜"的特徵表示為"芳"、"瑜"，以及原始的名子"芳瑜"。
最後我們來看看性別混淆的名子:
ind1 = (sex_tend['sex-mean'] > 0.4) & (sex_tend['sex-mean'] < 0.6)
sex_amb = sex_tend[ind1]
amb_count = sex_amb['sex-count'].sum()
print("難區分性別姓名數量: ", amb_count, "比率:", amb_count/ nobs)
難以區分性別的名子總數並不高，所占總體資料的比率也很低，只有1.26%，因此不用擔心這個因素會影響預測準確率。
下面來看看幾個容易混淆性別的名子:
sex_amb.head()
實做限制¶
為了確保可以確實練習到重要技巧，禁用Pycaret (https://pycaret.org/) 這類可以自動化調教參數與Ensemble模型的工具。另外也禁止使用sklearn.ensemble.Stacking.*。你可以使用sklearn中Pre-processing工具函數與現成的分類器，但參數調教請自行處理。
回答下面問題¶
Q1.1 (10%):¶
使用One-hot Encoding建構資料變數。所有資料變數都要是Numpy Array。依照每筆資料其fold值切割為Training (fold <=6)、Validation (fold == 7)、Stacking (fold == 8)、Test (fold == 9)。每個資料集應有特徵Array(二維)以及Label Array(一維)。如前面提到的，每個名子應該要對應到全名以及單字的One-hot Encoding。比如說，"芳瑜"的特徵表示為"芳"、"瑜"，以及原始的名子"芳瑜"。建構特徵表示時應依照個特徵出現的頻率篩選。特徵在訓練資料出現兩次或以上才納入。如果一個特徵被排除，這個特徵出現時應被歸為"_OtherFeature"。任何名子只要有出現未被納入的特徵，則其"_OtherFeature"的欄位值為1。
舉例而言，假設訓練資料集中有下面三個名子: 承恩、承德、恩賜。在經過特徵頻率篩選之後，只剩下以下特徵: 承、恩。其他特徵，如承恩、承德、恩賜、德、賜皆被排除。因此最後的特徵表示為:
| Input | 承 | 恩 | _OtherFeature | |
|---|---|---|---|---|
| 承恩 | 1 | 1 | 1 | |
| 承德 | 1 | 0 | 1 | |
| 恩賜 | 0 | 1 | 1 |
最後應產生以下Numpy Array:
- x_train: Training Features
- y_train: Training Labels
- x_valid: Validation Features
- y_valid: Validation Labels
- x_stack: Stack Features
- y_stack: Stack Labels
- x_test: Test Features
- y_test: Test Labels
請列出每個Numpy Array的Shape以資查驗。
nameintraining_df=name_df[name_df['fold']<=6]["name"]
nameintraining_df=nameintraining_df.reset_index(drop=True)
#nameintraining_df
word_dictionary_uni=""
# onlyname_df=name_testing
# onlyname_df=name_df["name"]
onlyname_df=nameintraining_df
for i in range(len(onlyname_df)):
for j in range(len(onlyname_df[i])):
compare_str=onlyname_df[i][j]
if compare_str not in word_dictionary_uni:
word_dictionary_uni=word_dictionary_uni+compare_str
word_dictionary_uni_list=[]
for i in range(len(word_dictionary_uni)):
word_dictionary_uni_list.append(word_dictionary_uni[i])
#print(word_dictionary_uni_list)
count=0
rareword_list_index=[]
for p in range(len(word_dictionary_uni_list)):
for q in range(len(onlyname_df)):
for r in range(len(onlyname_df[q])):
if word_dictionary_uni_list[p]==onlyname_df[q][r]:
count=count+1
if count>1:
count=0
break;
if count==1:
rareword_list_index.append(p)
count=0
#rareword_list_index
rareword_list=[]
for i in range(len(rareword_list_index)):
rareword_list.append(word_dictionary_uni_list[rareword_list_index[i]])
for i in range(len(rareword_list)):
word_dictionary_uni_list.remove(rareword_list[i])
unit_item=word_dictionary_uni_list
#unit_item
word_dictionary_uni_count_list=[]
count=0
for p in range(len(word_dictionary_uni_list)):
for q in range(len(onlyname_df)):
for r in range(len(onlyname_df[q])):
if word_dictionary_uni_list[p]==onlyname_df[q][r]:
count=count+1
word_dictionary_uni_count_list.append(count)
count=0
#print(len(word_dictionary_uni_count_list))
#word_dictionary_uni_count_list
name_onlytrain_df=name_df[name_df['fold']<=6]
sex_tend = name_onlytrain_df[['name', 'sex']].groupby(["name"]).agg(['mean', 'count']).reset_index()
sex_tend.columns = ['-'.join(col).strip() for col in sex_tend.columns.values]
sex_tend = sex_tend.sort_values(['sex-count'], ascending=False)
fullname_item=sex_tend[sex_tend['sex-count']>1]['name-']
fullname_item=fullname_item.tolist()
#print(len(fullname_item))
#fullname_item
clean_data_df=name_df
clean_data_df=pd.DataFrame(clean_data_df, columns=['name','fold','sex'])
clean_data_df=clean_data_df.rename({'name':'Input'}, axis='columns')
for i in range(len(unit_item)):
clean_data_df[unit_item[i]]=0
for i in range(len(fullname_item)):
clean_data_df[fullname_item[i]]=0
clean_data_df['_Other_Feature_']=0
clean_data_df
for i in range(len(clean_data_df)): #len(clean_data_df)
full_name=clean_data_df['Input'][i]
if(len(full_name)==1):
if full_name in clean_data_df.columns:
clean_data_df[full_name][i]=1
else:
clean_data_df['_Other_Feature_'][i]=1
else:
first=full_name[0]
second=full_name[1]
if full_name in clean_data_df.columns:
clean_data_df[full_name][i]=1
else:
clean_data_df['_Other_Feature_'][i]=1
if first in clean_data_df.columns:
clean_data_df[first][i]=1
else:
clean_data_df['_Other_Feature_'][i]=1
if second in clean_data_df.columns:
clean_data_df[second][i]=1
else:
clean_data_df['_Other_Feature_'][i]=1
clean_data_df
feature_name=clean_data_df.columns
feature_name=feature_name.delete([0,1,2])
feature_name
x_train=clean_data_df[clean_data_df['fold']<=6].drop(columns=['Input','fold','sex']).to_numpy()
y_train=clean_data_df[clean_data_df['fold']<=6]['sex'].to_numpy()
print("x_train shape:",x_train.shape,"; y_train shape:",y_train.shape)
x_valid=clean_data_df[clean_data_df['fold']==7].drop(columns=['Input','fold','sex']).to_numpy()
y_valid=clean_data_df[clean_data_df['fold']==7]['sex'].to_numpy()
print("x_valid shape:",x_valid.shape,"; y_valid shape:",y_valid.shape)
x_stack=clean_data_df[clean_data_df['fold']==8].drop(columns=['Input','fold','sex']).to_numpy()
y_stack=clean_data_df[clean_data_df['fold']==8]['sex'].to_numpy()
print("x_stack shape:",x_stack.shape,"; y_stack shape:",y_stack.shape)
x_test=clean_data_df[clean_data_df['fold']==9].drop(columns=['Input','fold','sex']).to_numpy()
y_test=clean_data_df[clean_data_df['fold']==9]['sex'].to_numpy()
print("x_test shape:",x_test.shape,"; y_test shape:",y_test.shape)
1.1 解釋：
共有1630個feature，shape如上程式輸出所示。
Q1.2 (10%):¶
使用sklearn.linear_model.LogisticRegression()建構Logistic Regression分類模型。利用Training與Validation調教Regularization Coefficient $c$。Grid Search的小值為$10^{-4}$，最大值是$1,000$，總共取20個點，使用等比級數間隔。取F-1 Score最大之Regularization Coefficient，印出此數值(以下稱bestc)。將Training與Validation合併後，令Regularizaiton Coefficient為bestc，訓練最終模型，並報告Test Data的Accuracy, Precision, Recall, 與F-1 Score。另外列出係數絕對值最大的20個特徵。注意應列出未取絕對值的係數數值，方知特徵的性別傾向。討論Prediction Performance與重要特徵的合理性。
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score
ratio=pow(1000/0.0001,1.0/19)
grids=[]
current=0.0001
grids.append(current)
for i in range(1,20):
current=current*ratio
grids.append(current)
bestf1=0
bestc=grids[0]
for i in range(len(grids)):
lrg=LogisticRegression(solver='liblinear',C=grids[i])
lrg.fit(x_train,y_train)
ypred=lrg.predict(x_valid)
f1=f1_score(y_valid,ypred)
print("C:",grids[i],"; F1:",f1)
if(f1>bestf1):
bestf1=f1
bestc=grids[i]
print("Best F1 Score:",bestf1,"; Best C:",bestc)
big_x_train=np.concatenate((x_train,x_valid),axis=0)
big_y_train=np.concatenate((y_train,y_valid),axis=0)
lrg=LogisticRegression(solver='liblinear',C=bestc)
lrg.fit(big_x_train,big_y_train)
ypred=lrg.predict(x_test)
accuracy=accuracy_score(y_test,ypred)
precision=precision_score(y_test,ypred)
recall=recall_score(y_test,ypred)
f1=f1_score(y_test,ypred)
print("C:",bestc,"\nAccuracy:",accuracy,"\nPrecision:",precision,"\nRecall:",recall,"\nF1:",f1)
coef=lrg.coef_[0]
coef_abs=np.absolute(coef)
sort_coef_index=np.argsort(-coef_abs)
top20_coef_index=sort_coef_index[0:20]
for i in range(len(top20_coef_index)):
print("Word:",feature_name[top20_coef_index[i]],coef[top20_coef_index[i]])
1.2 解釋：
表現最好的C為78.4759970351461，最終模型的表現上，F1約為0.87613，其餘指標如上輸出所示。係數特徵的部分，絕對值最大的是「傑」，第二是「森」，其餘可見於上輸出。
performance的表現上，可見應該還是有一成多一點的人的姓名沒有辦法被正確預測。但從重要特徵來看，大部分確實與我平常藉由名字判斷性別的印象差不多，例如看到「傑」，很容易就會覺得這是男生的名字（在特徵的係數上也是正數，代表此性別傾向男生），又或者是像「玲」，也會覺得是女生的名字（在特徵的係數上也是負數，代表此性別傾向女生）。
Q1.3 (10%):¶
使用sklearn.ensemble.RandomForestClassifier()建構Random Forest分類模型。利用Training與Validation調教Number of Tress (i.e., n_estimators)。Grid Search的小值為$5$，最大值是$1,000$，總共取10個點，使用等比級數間隔。取F-1 Score最大之n_estimators，印出此數值(best_nest)。將Training與Validation合併後，令n_estimators為best_nest，訓練最終模型，並報告Test Data的Accuracy, Precision, Recall, 與F-1 Score。另外列出最重要的20個特徵。討論Prediction Performance與重要特徵的合理性。
from sklearn.ensemble import RandomForestClassifier
ratio=pow(1000/5,1.0/9)
grids=[]
current=5
grids.append(current)
for i in range(1,10):
current=current*ratio
grids.append(round(current))
bestf1=0
best_nest=grids[0]
for i in range(len(grids)):
rf=RandomForestClassifier(max_depth=grids[i],random_state=543)
rf.fit(x_train,y_train)
ypred=rf.predict(x_valid)
f1=f1_score(y_valid,ypred)
print("N_est:",grids[i],"; F1:",f1)
if(f1>bestf1):
bestf1=f1
best_nest=grids[i]
print("Best F1 Score:",bestf1,"; Best nest :",best_nest)
rf=RandomForestClassifier(max_depth=best_nest,random_state=87)
rf.fit(big_x_train,big_y_train)
ypred=rf.predict(x_test)
accuracy=accuracy_score(y_test,ypred)
precision=precision_score(y_test,ypred)
recall=recall_score(y_test,ypred)
f1=f1_score(y_test,ypred)
print("Best nest:",best_nest,"\nAccuracy:",accuracy,"\nPrecision:",precision,"\nRecall:",recall,"\nF1:",f1)
importance=rf.feature_importances_
sort_coef_index=np.argsort(-importance)
top20_coef_index=sort_coef_index[0:20]
for i in range(len(top20_coef_index)):
print("Word:",feature_name[top20_coef_index[i]],importance[top20_coef_index[i]])
1.3 解釋：
表現最好的n_estimators為308，最終模型的表現上，F1約為0.8431，其餘指標如上輸出所示。係數絕對值特徵的部分，絕對值最大的是「妤」，第二是「翔」，其餘可見於上輸出。
performance的表現上，random forest的分類表現比logistic regression來得差，約有一成五的名字沒辦法被正確分類。從重要特徵來看，也大部分與我平常藉由名字判斷性別的印象差不多，但random forest列出的特徵沒有辦法藉由正負來看出是男生還是女生的傾向，但因為都是比較重要的特徵，所以也可以大致猜出性別。
Q1.4 (10%):¶
使用sklearn.ensemble.GradientBoostingClassifier()建構Gradient Boosting Decision Tree (GBDT)分類模型。利用Training與Validation調教learning_rate與n_estimators。考慮以下Learning Rate: 0.1, 0.5, 1。每一個Learning Rate設n_estimator為1,500並估計一個GBDT分類器，計算1,500個Stages下Validation F-1 Score值，繪圖，並找出讓F-1 Score最大的Stage與F-1 Score最大值。對所有Learning Rate重複同樣程序，找出最佳的Learning Rate (稱best_lr) 與其對應的Number of Stages(best_nstg)。將Training與Validation合併後，令learning_rate為best_lr, n_estimators為best_nstg，訓練最終模型，並報告Test Data的Accuracy, Precision, Recall, 與F-1 Score。另外列出最重要的20個特徵。討論Prediction Performance與重要特徵的合理性。
from sklearn.ensemble import GradientBoostingClassifier
import matplotlib.pyplot as plt
%matplotlib inline
lr_grids=[0.1, 0.5, 1]
best_stage_list=[]
best_f1_list=[]
for i in range(len(lr_grids)):
print("Learning Rate:",lr_grids[i])
GBDT=GradientBoostingClassifier(n_estimators=1500,learning_rate=lr_grids[i])
GBDT.fit(x_train,y_train)
beststage=0
bestf1=0
f1_score_list=[]
iteration=0
for ypred in GBDT.staged_predict(x_valid):
f1=f1_score(y_valid,ypred)
# print(f1)
f1_score_list.append(f1)
if f1>bestf1:
bestf1=f1
beststage=iteration
iteration=iteration+1
best_stage_list.append(beststage)
best_f1_list.append(bestf1)
plt.plot(f1_score_list)
plt.xlabel('Stages')
plt.ylabel('F1 Score')
title='F1_Score v.s. Stage (lr='+str(lr_grids[i])+')'
plt.title(title)
plt.show()
print(best_stage_list)
print(best_f1_list)
max_f1_value=max(best_f1_list)
max_lr_index=best_f1_list.index(max_f1_value)
best_lr=lr_grids[max_lr_index]
best_nstg=best_stage_list[max_lr_index]
print("Best Learning Rate:",best_lr)
print("Best Number of Stages:",best_nstg)
GBDT=GradientBoostingClassifier(n_estimators=best_nstg,learning_rate=best_lr)
GBDT.fit(big_x_train,big_y_train)
ypred=GBDT.predict(x_test)
accuracy=accuracy_score(y_test,ypred)
precision=precision_score(y_test,ypred)
recall=recall_score(y_test,ypred)
f1=f1_score(y_test,ypred)
print("Best lr:",best_lr,"Best nstg:",best_nstg,"\nAccuracy:",accuracy,"\nPrecision:",precision,"\nRecall:",recall,"\nF1:",f1)
importance=GBDT.feature_importances_
sort_coef_index=np.argsort(-importance)
top20_coef_index=sort_coef_index[0:20]
for i in range(len(top20_coef_index)):
print("Word:",feature_name[top20_coef_index[i]],importance[top20_coef_index[i]])
1.4 解釋：
表現最好的learning rate是0.1，最好的stage是第1083個stage。最終模型的表現上，F1約為0.86444，其餘指標如上輸出所示。係數絕對值特徵的部分，絕對值最大的是「妤」，第二是「翔」，其餘可見於上輸出。
performance的表現上，GBDT的分類表現比logistic regression來得差，但比random forest來得好。從重要特徵來看，也大部分與我平常藉由名字判斷性別的印象差不多，但GBDT列出的特徵沒有辦法藉由正負來看出是男生還是女生的傾向，但因為都是比較重要的特徵，所以也可以大致猜出性別。
Q1.5 (10%):¶
取用前面所建構的Logistic Regression, Random Forest, 與Gradient Boosting Decision Tree, 組合(Stacking)成一個新的分類器。我們使用Logistic Regression without Penalty建構這個Stacking分類器。訓練資料為Stacking資料集。各分類器輸入的特徵為男生預測機率(注意: 不是分類結果)。列出Stacking分類器的係數，討論係數的意義，並計算這個Stacking分類器在Test資料集的預測能力。
lrg_ypredprob=lrg.predict_proba(x_stack)
lrg_male_prob=lrg_ypredprob[:,1]
rf_ypredprob=rf.predict_proba(x_stack)
rf_male_prob=rf_ypredprob[:,1]
GBDT_ypredprob=GBDT.predict_proba(x_stack)
GBDT_male_prob=GBDT_ypredprob[:,1]
x_metatrain=([lrg_male_prob,rf_male_prob,GBDT_male_prob])
x_metatrain=np.stack(x_metatrain,axis=1)
x_metatrain
lrg_ypredprob=lrg.predict_proba(x_test)
lrg_male_prob=lrg_ypredprob[:,1]
rf_ypredprob=rf.predict_proba(x_test)
rf_male_prob=rf_ypredprob[:,1]
GBDT_ypredprob=GBDT.predict_proba(x_test)
GBDT_male_prob=GBDT_ypredprob[:,1]
x_metatest=([lrg_male_prob,rf_male_prob,GBDT_male_prob])
x_metatest=np.stack(x_metatest,axis=1)
x_metatest
lrg_meta=LogisticRegression(penalty='none')
lrg_meta.fit(x_metatrain,y_stack)
ypred=lrg_meta.predict(x_metatest)
accuracy=accuracy_score(y_test,ypred)
precision=precision_score(y_test,ypred)
recall=recall_score(y_test,ypred)
f1=f1_score(y_test,ypred)
print("Accuracy:",accuracy,"\nPrecision:",precision,"\nRecall:",recall,"\nF1:",f1)
lrg_meta.coef_
1.5 解釋：
Stacking的表現優於上述三個模型，F1表現為0.8806，其餘表現指標如上輸出所示。
分類器的係數部分，logistic regression的係數是2.94468097，random forest的則是-0.26344434，GBDT的則是5.33358152。由係數可以得知，在stacikng的分類器當中，GBDT對結果的影響是最顯著的，而random forest則是影響最微弱的。正負號來看，random forest對最後的結果有負向的影響，與其他兩個模型較為不同，但因為值蠻接近0的，所以影響應該不會到太大。
第二題 [Data Visualization via Dimensionality Reduction]¶
Data visualization is a useful technique that can help us understand the characteristics of a dataset. We are going to practice this skill using the University Offer of Admission Dataset (UDOAD).
Dataset: University Department Offer of Admission Dataset (UDOAD; 大學推薦甄選資料集)¶
Many high school students get admitted to universities through an application and screening process that requires each university department to offer admission to applicants first before students can choose where they want to go. Suppose we think of applicants as the customers of an academic department. In that case, the duplications of offered applicants from different departments can be used to understand the competitive relationships between academic departments. We are going to visualize this competition relationship using UDOAD.
UDOAD was collected through a popular online offer searching service (https://freshman.tw/; site no longer operational) for the 2017 academic year. We collected the offers received by each applicant as well as the basic information for academic departments. This dataset contains two files
- student_admission106.csv: the offers received by each student applicant.
- univ_name106short1.csv: the basic information, such as the name and field of academic departments.
Below is the first few records of these two files:
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
stu_adm = pd.read_csv('ds/student_admission106.csv', encoding="utf-8", dtype=str)
uname = pd.read_csv('ds/univ_name106short1.csv', encoding="utf-8", dtype=str)
all_depid = stu_adm['department_id'].unique()
all_stuid = stu_adm['student_id'].unique()
ndepid = all_depid.shape[0]
nstuid = all_stuid.shape[0]
print("There are %d students and %d departments in total." % (nstuid, ndepid))
print("offers received by students:")
stu_adm.head(10)
The department_id can uniquely identify an academic department. We do not care about the ranking of admission here, and you should ignore the "state" column. We only care about the "co-application" relations in this dataset. You should use student_id to identify a student applicant uniquely.
print("academic department basic information:")
uname.head(10)
You can use this dataset to identify the name of a department_id. The school_name and department_name contain the "full name" of an academic department. To facilitate visualization, we also provide "shorter names" in school_name_abbr and department_name_abbr. The category_name is the field of an academic department. This field is essential in our visualization exercise since you should color each data point according to its category_name.
Implementation Restrictions¶
You are allowed to use any Python libraries in your solution.
Data Preparation¶
Our focus is on the relationships between departments. To do this, we need to convert the raw data into a "matrix" representation. Each row represents an academic department, and each column represents a student applicant. The cell's value is one if a student applied for admission to the corresponding academic department, and 0 otherwise.
To avoid potential numerical problems, we only include an academic department if it received ten or more applications. Moreover, we only include a student applicant if he or she applied for more than one academic department.
Note that the two conditions should be satisfied "as is" in after preprocessing. For example, suppose a student applied for two departments in the original dataset, and one of the departments was removed. In that case, this student should be removed because the student only applied for one department in the processed dataset.
%matplotlib inline
import numpy as np
import pandas as pd
stu_adm = pd.read_csv('ds/student_admission106.csv', encoding="utf-8", dtype=str)
uname = pd.read_csv('ds/univ_name106short1.csv', encoding="utf-8", dtype=str)
all_depid = stu_adm['department_id'].unique()
all_stuid = stu_adm['student_id'].unique()
ndepid = all_depid.shape[0]
nstuid = all_stuid.shape[0]
print("In raw data, there are %d students and %d departments in total." % (nstuid, ndepid))
#construct the department-student matrix (i.e. array).
dep_stu = np.zeros((ndepid, nstuid))
rowname = all_depid.copy()
depid_seq_map = dict()
for i in range(ndepid):
depid_seq_map[all_depid[i]] = i
stuid_seq_map = dict()
for i in range(nstuid):
stuid_seq_map[all_stuid[i]] = i
for cindex, row in stu_adm.iterrows():
#print(cindex, row)
dep_seq = depid_seq_map[row['department_id']]
stu_seq = stuid_seq_map[row['student_id']]
#print(dep_seq, stu_seq)
dep_stu[dep_seq, stu_seq] = 1
#Remove very small departments.
min_stu_per_dep = 10
min_apply_dep_per_stu = 2
#remove small departments and single-application students.
dep_apply_sum = np.sum(dep_stu, axis = 1)
keeprow = dep_apply_sum >= min_stu_per_dep
rowname = rowname[keeprow]
dep_stu2 = dep_stu[keeprow,:]
stu_apply_sum = np.sum(dep_stu2, axis = 0)
dep_stu2 = dep_stu2[:, stu_apply_sum >= min_apply_dep_per_stu]
#another run of filtering
dep_apply_sum = np.sum(dep_stu2, axis = 1)
dep_stu2 = dep_stu2[dep_apply_sum >= min_stu_per_dep,:]
rowname = rowname[dep_apply_sum >= min_stu_per_dep]
stu_apply_sum = np.sum(dep_stu2, axis = 0)
dep_stu2 = dep_stu2[:, stu_apply_sum >= min_apply_dep_per_stu]
#third run of filtering
dep_apply_sum = np.sum(dep_stu2, axis = 1)
dep_stu2 = dep_stu2[dep_apply_sum >= min_stu_per_dep,:]
rowname = rowname[dep_apply_sum >= min_stu_per_dep]
stu_apply_sum = np.sum(dep_stu2, axis = 0)
dep_stu2 = dep_stu2[:, stu_apply_sum >= min_apply_dep_per_stu]
#check to make sure the two conditions are satisfied.
dep_apply_sum = np.sum(dep_stu2, axis = 1)
print("Number of department too small:", np.sum(dep_apply_sum < min_stu_per_dep))
stu_apply_sum = np.sum(dep_stu2, axis = 0)
print("Number of students applying only one department:", np.sum(stu_apply_sum <min_apply_dep_per_stu))
#now both conditions are satisfied.
Report the number of academic departments and student applicants in the processed dataset:
num_dep, num_stu = dep_stu2.shape
print("In final dataset, there are", num_dep, "academic departments and", num_stu, "student applicants.")
Report the top ten departments that received the most applications and the number of applications they received:
uname['depname'] = uname.school_name_abbr + uname.department_name_abbr
uname2 = uname[['department_id', 'depname', 'category_name']].copy()
#this is for later use, to color data points.
category_id, category_levels = pd.factorize(uname2.category_name)
#uname2['category_id'] = category_id / np.max(category_id)
uname2['category_id'] = category_id
#create a data frame for column name
colname_df = pd.DataFrame({'department_id': rowname})
colname_df = colname_df.merge(uname2, how = "left", on="department_id")
topdepid = np.argsort(dep_apply_sum)[::-1]
topn = 10
topdep = pd.DataFrame({'department_id': rowname[topdepid[0:topn]],
'department_name': colname_df.depname.values[topdepid[0:topn]],
'num_applicant': dep_apply_sum[topdepid[0:topn]]
})
topdep
Answer the following questions¶
Visualize academic departments in the following questions. In all plots, you should color data points according to the academic department's category. Moreover, you should provide a legend or a picture that illustrates the mapping between colors and category names. Visualize the data using two-dimensional plots. Note that it is your responsibility to study the documentation of your choice libraries and make sure that the results are reasonable.
- Q2.1 (10%): Visualize academic departments using the first eight principal components. Use your judgment to select multiple pairs of principal components to visualize. Discuss the visual patterns concerning department categories.
- Q2.2 (10%): Visualize academic departments using multiple dimensional scaling. Consider both the metric and non-metric settings. Discuss the result.
- Q2.3 (10%): Visualize academic departments using Locally Linear Embedding. Consider three variations: (1) Use 20 neighbors to construct the weight matrix; (2) Use 40 neighbors to construct the weight matrix; (3) Perform PCA transformation first, and use the first 100 principal components as the input to LLE (with 20 neighbors). Discuss the result.
- Q2.4 (10%): Visualize academic departments using Kernel PCA. You should at least consider the RBF and Cosine kernel. It is your responsibility to select reasonably good kernel parameters. Discuss the result.
- Q2.5 (10%): Visualize academic departments using t-SNE. You should consider at least the Euclidian, Cosine, and Jaccard metric. Set numpy random seed so that your results can be repeated. Discuss the result.
- Q2.6 (10%) Select the most promising visualization method in the previous question and refine the result. You should color points by department category. Label each data point with its name so that we can quickly identify a data point on the picture. Moreover, you should try to reduce the problem caused by overlapping points and labels. Output an image that is large enough so that a user can easily identify a department and its neighbors. Jupyter Lab has limitations on the largest picture size. To overcome this problem, output the picture to a separate file and submit the file for grading. Your score depends on how useful, readable, and visually pleasing of your visualization results.
filter_mask=uname['department_id'].isin(rowname)
department_info=uname[filter_mask]
department_info.head(5)
dep_stu_categoryname_list=[]
for i in range(len(department_info)):
tempname=department_info[department_info['department_id']==rowname[i]]['category_name']
dep_stu_categoryname_list.append(tempname)
dep_stu_categoryname=np.array(dep_stu_categoryname_list)
dep_stu_depname_list=[]
for i in range(len(department_info)):
tempname=department_info[department_info['department_id']==rowname[i]]['depname']
tempname=tempname.to_string(index=False)
dep_stu_depname_list.append(tempname)
dep_stu_depname=np.array(dep_stu_depname_list)
Q2.1 (10%)¶
Visualize academic departments using the first eight principal components. Use your judgment to select multiple pairs of principal components to visualize. Discuss the visual patterns concerning department categories.
# Q2.1
from sklearn.decomposition import PCA
pca=PCA(n_components=8)
pca.fit(dep_stu2)
pca_outcome=pca.transform(dep_stu2)
print("Variance:",pca.explained_variance_ratio_)
pca_withlabel=np.c_[pca_outcome,dep_stu_categoryname]
pca_withlabel=pd.DataFrame(pca_withlabel)
columnname=['PC_1','PC_2','PC_3','PC_4','PC_5','PC_6','PC_7','PC_8','Category']
pca_withlabel.columns=columnname
pca_withlabel
from matplotlib import font_manager
my_font=font_manager.FontProperties(fname="msjhl.ttc")
groups=pca_withlabel['Category'].unique()
colors_list=['red','orange','yellow','green','blue','purple','darkgoldenrod','black','fuchsia','cyan','lime']
def drawscatterplot(df_fitoutcome):
df_withlabel=np.c_[df_fitoutcome,dep_stu_categoryname]
df_withlabel=pd.DataFrame(df_withlabel)
columnname=['Coordinate1','Coordinate2','Category']
df_withlabel.columns=columnname
display(df_withlabel.head(5))
for i in range(len(groups)):
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
plt.figure(figsize=(60,60))
subplot_num=1
for p in range(len(columnname[:8])):
for q in range(len(columnname[:8])):
plt.subplot(8,8,subplot_num)
for i in range(len(groups)):
x=pca_withlabel[pca_withlabel['Category']==groups[i]][columnname[p]]
y=pca_withlabel[pca_withlabel['Category']==groups[i]][columnname[q]]
plt.scatter(x,y,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel(columnname[p])
plt.ylabel(columnname[q])
subplot_num=subplot_num+1
plt.rcParams["figure.dpi"]=200
for i in range(len(groups)):
x=pca_withlabel[pca_withlabel['Category']==groups[i]]['PC_1']
y=pca_withlabel[pca_withlabel['Category']==groups[i]]['PC_2']
plt.scatter(x,y,s=10,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel('PC_1')
plt.ylabel('PC_2')
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
for i in range(len(groups)):
x=pca_withlabel[pca_withlabel['Category']==groups[i]]['PC_3']
y=pca_withlabel[pca_withlabel['Category']==groups[i]]['PC_4']
plt.scatter(x,y,s=10,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel('PC_3')
plt.ylabel('PC_4')
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
2.1 解釋：
我有針對前八個PC繪製出不同組合的散佈圖，雖然從這幾張輸出的圖來看，好像大多數的點點都擠在一起，沒有辦法看出很明顯的區隔，不過其中我認為PC1和PC2的組合以及PC3以及PC4的組合比較可以看出類別之間的分別，圖表如上輸出所示。
其中，有蠻多工程製造及營建（藍色）相關的科系會在PC1的值比較大，而也有不少醫藥衛生及社會福利的相關科系（綠色）則是在PC2的值有比較大的傾向。
另外，亦可從PC3和PC4的散佈圖中看出，藝術與人文（紅色）的科系有頗多落在PC3值小於0的位置。至於PC4，可看出商業管理及法律（土黃色）的科系，會有較大的PC4值，而工程製造及營建（藍色）則會有較小的PC4值。
Q2.2 (10%)¶
Visualize academic departments using multiple dimensional scaling. Consider both the metric and non-metric settings. Discuss the result.
# Q2.2
from sklearn.manifold import MDS
# metric
mds_metric=MDS(n_components=2,metric=True)
mds_outcome_metric=mds_metric.fit_transform(dep_stu2)
drawscatterplot(mds_outcome_metric)
# non-metric
mds_nonmetric=MDS(n_components=2,metric=False)
mds_outcome_nonmetric=mds_nonmetric.fit_transform(dep_stu2)
drawscatterplot(mds_outcome_nonmetric)
2.2 解釋：
在multiple dimensional scaling這個方法的表現上，很難可以從散佈圖中看出什麼pattern，大部分的資料點都是混雜在一起，但整體可看出non-metric的散佈圖可能比metric的還要來得分散一點，點與點之前的重疊也稍微少一些。但儘管如此，此方法對於這筆資料集、並用這10個類別來區分的情況下，降維效果並不是太理想。
Q2.3 (10%)¶
Visualize academic departments using Locally Linear Embedding. Consider three variations: (1) Use 20 neighbors to construct the weight matrix; (2) Use 40 neighbors to construct the weight matrix; (3) Perform PCA transformation first, and use the first 100 principal components as the input to LLE (with 20 neighbors). Discuss the result.
# Q2.3
from sklearn.manifold import LocallyLinearEmbedding
# (1) 20 neighbors
LLE_20=LocallyLinearEmbedding(n_neighbors=20,n_components=2,random_state=0)
LLE_20_outcome=LLE_20.fit_transform(dep_stu2)
drawscatterplot(LLE_20_outcome)
# 20 neighbors closer
df_fitoutcome=LLE_20_outcome
df_withlabel=np.c_[df_fitoutcome,dep_stu_categoryname]
df_withlabel=pd.DataFrame(df_withlabel)
columnname=['Coordinate1','Coordinate2','Category']
df_withlabel.columns=columnname
display(df_withlabel.head(5))
for i in range(len(groups)):
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.xlim(-0.002,-0.0005)
plt.ylim(-0.005,0.005)
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
# (2) 40 neighbors
LLE_40=LocallyLinearEmbedding(n_neighbors=40,n_components=2)
LLE_40_outcome=LLE_40.fit_transform(dep_stu2)
drawscatterplot(LLE_40_outcome)
# 40 neighbors closer
df_fitoutcome=LLE_40_outcome
df_withlabel=np.c_[df_fitoutcome,dep_stu_categoryname]
df_withlabel=pd.DataFrame(df_withlabel)
columnname=['Coordinate1','Coordinate2','Category']
df_withlabel.columns=columnname
display(df_withlabel.head(5))
for i in range(len(groups)):
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.xlim(-0.0025,0.0025)
plt.ylim(-0.005,0.002)
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
# (3) PCA + 20 neighbors
pca=PCA(n_components=100)
pca.fit(dep_stu2)
pca_outcome=pca.transform(dep_stu2)
LLE_pca20=LocallyLinearEmbedding(n_neighbors=20,n_components=2)
LLE_pca20_outcome=LLE_pca20.fit_transform(pca_outcome)
drawscatterplot(LLE_pca20_outcome)
2.3 解釋：
從第(1)小題，neighbor=20的散佈圖來看，大多數的資料點都落在Coordiante1與Coordinate2的0到0.1之間。若是有資料點在Coordinate1的值大於0.1，則為商業管理及法律的科系；而若有資料點落在Coordinate2大於0.1的位置，則是醫藥衛生及社會福利的科系。若細部放大，讓Coordinae1介於(-0.002,-0.0005)，Coordinate2介於(-0.005,0.005)，依舊可以看出資料點大部分都擠在一起，並沒有明顯的區隔。
從第(2)小題，neighbor=40的散佈圖來看，如同(1)小題，大多數的資料點都落在Coordiante1的0到-0.1之間與Coordinate2的0到0.1之間。若是有資料點在Coordinate1的值小於-0.1，則為商業管理及法律的科系；而若有資料點落在Coordinate2大於0.1的位置，則是醫藥衛生及社會福利的科系。 若細部放大，讓Coordinae1介於(-0.0025,0.0025)，Coordinate2介於(-0.005,0.002)，儘管「其他」這個類別（亮藍色）可能看似有比較明顯的區隔，但整體類別仍舊是沒有太顯著的區隔。
從第(3)小題，取前100個PCA且neighbor=20的散佈圖來看，比較沒有都擠在很小的區域的狀況，大部分的資料點也比前兩題的圖分得更開，但整體依舊很難看出區別。可看出LLE在這份資料集、並用這10個類別來區分的情況下，也不是太好的降維方式。
Q2.4 (10%)¶
Visualize academic departments using Kernel PCA. You should at least consider the RBF and Cosine kernel. It is your responsibility to select reasonably good kernel parameters. Discuss the result.
# Q2.4
from sklearn.decomposition import KernelPCA
# RBF
rbf_pca=KernelPCA(n_components=2,kernel='rbf')
rbf_pca_outcome=rbf_pca.fit_transform(dep_stu2)
drawscatterplot(rbf_pca_outcome)
# Cosine
cosine_pca=KernelPCA(n_components=2,kernel='cosine')
cosine_pca_outcome=cosine_pca.fit_transform(dep_stu2)
drawscatterplot(cosine_pca_outcome)
# Poly
poly_pca=KernelPCA(n_components=2,kernel='poly')
poly_pca_outcome=poly_pca.fit_transform(dep_stu2)
drawscatterplot(poly_pca_outcome)
2.4 解釋：
這題我試了rbf、cosine以及poly三種kernal，僅能看出不管是哪種kernal，醫藥衛生及社會福利（綠色）有部分的科系資料點總是會有比較大的Cordiante1和Coordinate2值，而工程製造及營建這類別也有不少的科系資料點會落在Coordiante1值比較大的位置。
除此之外，這幾張圖的其他資料點大多擠在一起，看不太出明顯的區隔。
Q2.5 (10%)¶
Visualize academic departments using t-SNE. You should consider at least the Euclidian, Cosine, and Jaccard metric. Set numpy random seed so that your results can be repeated. Discuss the result.
# Q2.5
from sklearn.manifold import TSNE
# euclidean
tsne_euclidean=TSNE(n_components=2,metric='euclidean',random_state=87)
tsne_euclidean_outcome=tsne_euclidean.fit_transform(dep_stu2)
drawscatterplot(tsne_euclidean_outcome)
# jaccard
tsne_jaccard=TSNE(n_components=2,metric='jaccard',random_state=87)
tsne_jaccard_outcome=tsne_jaccard.fit_transform(dep_stu2)
drawscatterplot(tsne_jaccard_outcome)
# minkowski
tsne_minkowski=TSNE(n_components=2,metric='minkowski',random_state=87)
tsne_minkowski_outcome=tsne_minkowski.fit_transform(dep_stu2)
drawscatterplot(tsne_minkowski_outcome)
# cosine
tsne_cosine=TSNE(n_components=2,metric='cosine',random_state=87)
tsne_cosine_outcome=tsne_cosine.fit_transform(dep_stu2)
drawscatterplot(tsne_cosine_outcome)
# tsne cosine only pink class
df_fitoutcome=tsne_cosine_outcome
df_withlabel=np.c_[df_fitoutcome,dep_stu_categoryname]
df_withlabel=pd.DataFrame(df_withlabel)
columnname=['Coordinate1','Coordinate2','Category']
df_withlabel.columns=columnname
display(df_withlabel.head(5))
plt.rcParams["figure.dpi"]=1000
i=8
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=10)
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
# tsne cosine only blue class
plt.rcParams["figure.dpi"]=1000
i=4
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=2)
for i in range(df_withlabel.shape[0]):
if '電機' in dep_stu_depname[i]:
plt.text(x=df_withlabel.Coordinate1[i]+0.1,y=df_withlabel.Coordinate2[i]+0.1,s=dep_stu_depname[i], color='black',fontproperties=my_font)
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=10)
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
# tsne cosine five school label
plt.rcParams["figure.dpi"]=1000
for i in range(len(groups)):
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=2)
for i in range(df_withlabel.shape[0]):
if '臺大' in dep_stu_depname[i] or '清大' in dep_stu_depname[i] or '交大' in dep_stu_depname[i] or '成大' in dep_stu_depname[i] or '政大' in dep_stu_depname[i]:
plt.text(x=df_withlabel.Coordinate1[i]+0.1,y=df_withlabel.Coordinate2[i]+0.1,s=dep_stu_depname[i], color='black',fontproperties=my_font)
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=10)
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
2.5 解釋：
這題我試了euclidean、jaccard、minkowski與cosine四種metric。euclidean和minkowski的表現並不是太理想，很多資料點都擠在一起。
不過，jaccard、cosine的散佈圖則似乎隱約可看出不同顏色的點會一小群一小群的聚在一起，因此我針對metric設為cosine的T-SNE繪製更多圖片來觀察。舉例來說，若是僅單獨繪製「教育」這個類別的資料點，可以看出來大多數的資料點會大概分在兩群。分別是落在Coordinate1的0至-20與Coordinate2的0至(-20)之間一群；以及在Coordinate1的-60至-40與Coordinate2的30至40之間另一群。而若單獨繪製「工程、製造及營建」這個類別，則可以看出資料點大概會落在整張圖的右上和左下。由這種一小群一小群聚在一起的情況來看，這十個類別之中應該還有更細緻的次類別，例如物理系和化學系雖然都是「自然科學、數學及統計」這個大類別，但物理系自己會聚在一起，化學系會另外聚在一起，這兩小群也許在散佈圖上會在不同的位置。
另外，若是要再進一步探討，針對台清交成政五間學校的科系標註科系名稱至散佈圖中，可發現這幾間學校的科系的資料點大多都落在整張散佈圖的右下角，由此可知，或許（僅是針對散佈圖與標籤做粗略推論）使用學校作為顏色類別來繪製散佈圖，也可以看出一些明顯的區隔。
Q2.6 (10%)¶
Select the most promising visualization method in the previous question and refine the result. You should color points by department category. Label each data point with its name so that we can quickly identify a data point on the picture. Moreover, you should try to reduce the problem caused by overlapping points and labels. Output an image that is large enough so that a user can easily identify a department and its neighbors. Jupyter Lab has limitations on the largest picture size. To overcome this problem, output the picture to a separate file and submit the file for grading. Your score depends on how useful, readable, and visually pleasing of your visualization results.
# Q2.6
from sklearn.manifold import TSNE
plt.rcParams["figure.dpi"]=1000
tsne_cosine=TSNE(n_components=2,metric='cosine',random_state=87)
tsne_cosine_outcome=tsne_cosine.fit_transform(dep_stu2)
df_fitoutcome=tsne_cosine_outcome
df_withlabel=np.c_[df_fitoutcome,dep_stu_categoryname]
df_withlabel=pd.DataFrame(df_withlabel)
columnname=['Coordinate1','Coordinate2','Category']
df_withlabel.columns=columnname
display(df_withlabel.head(5))
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=0.5)
for i in range(len(groups)):
x=df_withlabel[df_withlabel['Category']==groups[i]][columnname[0]]
y=df_withlabel[df_withlabel['Category']==groups[i]][columnname[1]]
plt.scatter(x,y,s=8,alpha=0.6,color=colors_list[i],label=groups[i])
for i in range(df_withlabel.shape[0]):
plt.text(x=df_withlabel.Coordinate1[i]+0.1,y=df_withlabel.Coordinate2[i]+0.1,s=dep_stu_depname[i],
color='black',fontproperties=my_font)
my_font=font_manager.FontProperties(fname="msjhl.ttc",size=10)
plt.xlabel(columnname[0])
plt.ylabel(columnname[1])
plt.legend(prop=my_font,bbox_to_anchor=(1.05, 1.0),loc='upper left')
2.6 解釋：
我針對TSNE、metric為cosine的降維方法來繪製含有資料標籤（也就是學校科系名稱）的散佈圖，可見附檔tsne_cosine_all.png檔案。